Bias toward Higher Performance for the Decomposed Network. Whether a Decomposed Network Would Learn

faster than a non-decomposed network when the number of weights in the two networks is approximately equal remains to be tested. Second, we performed a more extensive search in and space than [Kamm and Singhal, 1990], resulting in generally higher learning rates and training times of only a few epochs. The range of learning rates explored in the current study included higher rates than those explored in [Kamm and Singhal, 1990], so it is dicult to determine whether the learning speed-up (in terms of arithmetic operations) is attributable to problem decomposition or learning rate. To address this issue, we also trained monolithic 125ms networks for approximately the same number of operations as the decomposed net, using a) the learning parameters used for the 125ms subnetwork and b) the original parameters for the monolithic network. On the 200-sentence-test set, these networks had normalized AHAs of 47.3% and 46.3% and normalized AFAs of 5.9% and 5.1%, respectively. This performance is markedly poorer than performance of the decomposed network, supporting the hypothesis that the problem decomposition strategy was a primary contributor to the learning speed-up. The problem decomposition strategy facilitated the more thorough search for learning parameters, since it allowed the use of dierent parameters for training the dierent subnetworks. In many empirical studies utilizing the back-propagation algorithm, it is dicult to control for the and parameters. Often, they are held constant over all experimental conditions. However, since optimal parameter values are often problem-dependent, this constraint can result in misleading comparisons if and are chosen optimally for one network but not another. Our experience in this project supports the fact that the careful selection of network training parameters is critical for successful training (and performance). 6 Summary Problem decomposition was applied to AP-net, a neural network for mapping acoustic spectra to phoneme classes. AP-net was decomposed into subnetworks diering in both output units (assigning phoneme classes to subnetworks based on their durations) and input units (providing dierent input spans to the subnetworks). Subnetworks were trained individually and then combined into a larger network using \glue" units and ne-tuning of the entire network. This problem decomposition strategy achieved comparable performance to that of a monolithic network, but required an order-of-magnitude fewer arithmetic operations during training. This result provides evidence for the general utility of this problem decomposition methodology for addressing the learning speed problem in back-propagation. Table 1 summarizes the conditions used to train …